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DESCRIPTION 



FLICKER REDUCTION METHOD, IMAGE PICKUP DEVICE AND FLICKER 

REDUCTION CIRCUIT 

5 

Technical Field 

The present invention relates to a method for reducing 
a fluorescent light flicker occurring in a video signal from 
an XY addressing type scanning image pickup element (an 

10 imager or an image sensor) such as a CMOS (complementary 
metal oxide semiconductor) image pickup element when a 
subject is photographed by the image pickup element under 
light of a fluorescent lamp, an image pickup device, such as 
a video camera or a digital still camera employing the XY 

15 addressing type scanning image pickup element such as the 
CMOS image pickup element, and a flicker reduction circuit 
for use in the image pickup device . 

Background Art 

20 When a subject is photographed by a video camera under 

an illumination of a fluorescent lamp directly operated from 
commercial AC power, chronological brightness level 
variations in a video signal as an image output, i.e., a 
fluorescent light flicker occurs due to a difference between 

25 the frequency (twice the frequency of the commercial AC 
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power) of a luminance change (light intensity change) of the 
fluorescent lamp and a vertical synchronization frequency of 
the camera - 

For example, in a zone where the frequency of the 
•5 commercial AC power is 50 Hz, a subject may be photographed 
by a CCD camera of an NTSC method (at a 60 Hz vertical 
synchronization frequency) under the illumination of the 
non- inverter fluorescent lamp. In such a case as shown in 
Fig. 28, one field frequency is 1/60 second while the period 

10 of the luminance change of the fluorescent lamp is 1/100 
second. The exposure timing at each field drifts with 
respect to variations in the luminance of the fluorescent 
lamp, and an amount of exposure light at each pixel changes 
from field to field. 

15 If the exposure time is 1/60 second, the amount of 

exposure light is different within the same exposure time 
from duration al to duration a2 to duration a3 , and when the 
exposure time is shorter than 1/60 second (but not l/lOO 
second) , the amount of exposure light time is different 

20 within the same exposure from duration bl to duration b2 to 
duration b3 . 

Since the exposure timing responsive to the luminance 
change of the fluorescent lamp reverts back to the original 
timing every three fields, the brightness level variation 
25 due to flickering is repeated every three fields. More 
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specifically, the luminance ratio of each field (the 
appearance of a flicker) changes within an exposure period, 
but the period of flickering remains unchanged. 

In a progressive camera such as a digital camera, the 
5 brightness level variation is repeated every three frames if 
the vertical synchronization frequency is 30 Hz. 

To emit white light, a plurality of fluorescent lamps, 
for example, a red fluorescent lamp, a green fluorescent 
lamp, and a blue fluorescent lamp are typically used. 

10 Fluorescent materials for these lamps have unique 

persistence characteristics thereof, and for a duration of 
time from a stop of discharging to a subsequent start of 
discharging, light emission decays in accordance with the 
persistence characteristics. During this duration of time, 

15 light appearing white at first decays while the hue thereof 
changes at the same time. If the exposure timing drifts, 
not only the brightness level variations but also the hue 
change occurs. Since the fluorescent lamp has unique 
spectral characteristics that a strong peak is present in a 

20 particular wavelength, a variable component of ^a signal 
becomes different from color to color. 

The color hue change and the difference in the variable 
component from color to color lead to a so-called color 
flicker. 

25 As shown in the bottom portion of Fig. 28, the amount 
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of exposure light remains constant regardless of the 
exposure timing if the exposure time is set to be an integer 
multiple of periods (1/100 second) of the luminance 
variation of the fluorescent lamp. No flicker then occurs. 
5 It is contemplated that illumination of the fluorescent 

lamp is detected through signal processing of a camera in 
response to an operation of a user, and that the exposure 
time is set to be an integer multiple of 1/100 second under 
the illumination of the fluorescent lamp. In this 

10 arrangement, a simple method can fully control the 
generation of the flickering. 

However, since this method does not allow the exposure 
time to be set to any value, the freedom of the exposure 
amount adjustment means for obtaining an appropriate amount 

15 of exposure is reduced. 

A method for reducing the fluorescent light flicker 
under any shutter speed (exposure time) is thus required. 

An image pickup device having all pixels in one frame 
exposed at the same exposure timing, such as a CCD Image 

2 0 pickup device, offers relatively easily such a method 

because the brightness level variations and color variations 
due to the flickering appears only between fields . 

If the exposure time is not l/lOO second, the 
flickering has a repetition frequency of three fields as 

25 shown in Fig. 28. To achieve a constant average value of 
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the video signal of each field, current luminance variations 
and current color variations are predicted from the video 
signal three fields before, and the gain of the video signal 
of each field is adjusted based on the prediction results so 
5 that the flicker is reduced to a level that presents no 
problem in practice . 

In the XY addressing type scanning image pickup device, 
such as a CMOS image pickup device, however, the exposure 
timing drifts successively from pixel to pixel by one 

10 horizontal period of the reading clock (pixel clock) in a 

screen horizontal direction. Since all pixels are different 
in the exposure timing, the above -referenced method cannot 
suppress the flickering. 

Fig. 29 illustrates such flickering. The pixels 

15 successively drift in the exposure timing in the screen 
horizontal direction as discussed above. One horizontal 
period is . sufficiently shorter than the period of the 
variation of the fluorescent light. Based on the assumption 
that the pixels on the same line have the same. timing, the 

20 exposure timing of each line in a screen vertical direction 
becomes something like the one shown in Fig. 29. This 
assumption presents no practical problem. 

The exposure timing is different from line to line in 
the XY addressing type scanning image pickup device, such as 

25 a CMOS image pickup device as shown in Fig. 29 (Fl 
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represents such a drift in the exposure timing) . Since the 
lines suffer from a difference in the ^amount of exposure 
light, the brightness level variation and the color 
variation take place due to the flickering not only between 
5 fields but also within each field. A strip pattern appears 
on a screen (with strips thereof aligned in the horizontal 
direction and the density of the stripes changing in the 
vertical direction) . 

Fig. 30 illustrates an on-screen flicker if a subject 

10 is a uniform pattern. Since one horizontal period (one 

wavelength) of the subject is 1/100 second, a stripe pattern 
of 1.666 periods appears in one frame. Let M represent the 
number of read lines per field, and one horizontal period of 
the stripe pattern corresponds to the number of read lines 

15 L=M*60/100- In this description and the drawing, an 
asterisk (*) represents a multiplication operation. 

As shown in Fig. 31, three fields (three frames) 
correspond to five periods (five wavelengths) of the stripe 
pattern, and if viewed continuously, the stripe pattern 

20 appears to drift in a vertical direction. 

Fig. 30 and Fig. 31 show only the brightness level 
variation due to the flicker. In practice, however, the 
above-described color variation also additionally appears , 
thereby substantially degrading image quality. The color 

25 flicker, in particular, becomes pronounced as the shutter 
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speed becomes fast . The XY addressing type scanning image 
pickup device suffers more from image quality degradation 
because the effect of the color flicker also appears on the 
screen . 

If the exposure timing is set to be an integer multiple 
of periods (l/lOO second) of the luminance variation of the 
fluorescent light, the amount of exposure becomes constant 
regardless of the exposure timing, and a fluorescent light 
flicker containing an on-screen flicker does not occur. 

With a variable electronic shutter speed feature 
incorporated, a CMOS image pickup device becomes complex in 
structure. Even in an image pickup device having the 
electronic shutter, the flexibility of the exposure amount 
adjusting means for achieving an appropriate exposure is 
reduced if only an integer multiple of 1/100 second is set 
as the exposure time to prevent flickering. 

Methods for reducing the fluorescent light flickering 
for use in the XY addressing type scanning image pickup 
device, such as the CMOS image pickup device, have toeen 
proposed. 

Patent document 1 (Japanese Unexamined Patent 
Application Publication No. 2000-350102) and patent document 
2 (Japanese Unexamined Patent Application Publication No. 
2000-23040) discloses methods of estimating a flicker 
component by measuring an amount of light of a fluorescent 
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lamp with a photosensitive element or a measuring element 
and controlling a gain of a video signal from an image 
pickup element in response to the estimation result. 
Patent document 3 (Japanese Unexamined Patent 
5 Application Publication No. 2001-16508) discloses another 
technique. In accordance with the disclosed technique, two 
types of images are taken in two conditions, namely, a first 
electronic shutter value appropriate for a current ambient 
illumination condition and in a second electronic shutter 

10 value having a predetermined relationship to a light and 
dark cycle of a fluorescent lamp, a flicker component is 
estimated by comparing the two signals, and a gain of a 
video signal from an image pickup device is controlled in 
response to the estimation results. 

15 Patent document 4 (Japanese Unexamined Patent 

Application Publication No. 11-164192) discloses another 
technique. In accordance with the disclosed technique, a 
■brightness variation under an illumination of a fluorescent 
lamp is recorded beforehand as a correction factor in a 

20 memory, the phase of a flicker component is detected from a 
video signal from an image pickup device taking aidvantage of 
a difference between the frequency of a video signal 
component and the frequency of the flicker component, and 
the video signal is thus corrected in accordance with the 

25 correction factor in the memory in response to the detection 
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results. 

Patent document 5 (Japanese Unexamined Patent 
Application Publication No. 2000-165752) discloses another 
technique. In the disclosed technique, a correction 
5 coefficient is calculated from two video signals that are 
obtained as a result of exposures performed with a time 
difference, the time difference causing the phase of flicker 
to be inverted by 180 degrees. 

As disclosed in patent documents 1 and 2, the technique 
10 of estimating of the flicker component by measuring the 
amount of light of the fluorescent lamp with the 
photosensitive element or the measuring element increases 
the size and the cost of the image pickup system because the 
photosensitive element or the measuring element is attached 
15 to the image pickup device . 

As disclosed in patent document 3, the technique of 
estimating the flicker component by photographing the two 
types of images in the different shutter conditions 
(exposure conditions) requires a complex . system in the image 
20 pickup device, and further this technique is not appropriate 
for taking a moving image . 

The technique disclosed in patent document 4 uses the 
coefficient prepared beforehand in the memory as a 
correction signal. It is practically impossible to -prepare 
25 the correction coefficients for all types of fluorescent 
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lamps. Depending on the type of the fluorescent lamp, 
detecting accurately the flicker component and reducing 
reliably the flicker component are difficult. As disclosed 
in patent document 4, the technique of extracting the 
5 flicker component from the video signal taking advantage of 
the difference between the frequencies of the video signal 
component and the flicker component has difficulty in 
detecting the flicker component distinctly from the video 
signal component in a black background portion and a low- 

10 illuminance portion, each portion having a small amount of 

flicker component. If a moving image is present in a screen, 
performance for detecting the flicker component is 
substantially lowered. 

As the technique disclosed in patent document 3, the 

15 technique disclosed in patent document 5 for estimating the 
flicker component by photographing the two types of images 
at the different timings requires a complex system in the 
image pickup device and is not appropriate for taking a 
moving image . 

20 In accordance with the present invention, a fluorescent 

light flicker characteristic of an XY addressing type 
scanning image pickup device such as a CMOS ima:ge pickup 
device is accurately detected and reliably and sufficiently 
reduced through simple signal processing without using an 

25 photosensitive element regardless of the level of a video 
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signal of a subject and the type of a fluorescent lamp. 

Disclosure of Invention 

A flicker reduction method of a first invention for 
5 reducing a fluorescent light flicker component in a video 
signal or a luminance signal obtained by photographing a 
subject through an XY addressing type image pickup element 
under an illumination of a fluorescent lamp, includes 

a step of integrating the video signal or the luminance 
10 signal, as an input image . signal, throughout a duration of 
time equal to or longer than one horizontal period, 

a step of normalizing the integrated value or a 
difference value between the integrated values of adjacent 
fields or adjacent frames, 
15 a step of extracting a spectrum of the normalized . 

integrated value or the normalized difference value, 

a step of estimating a flicker component from the 
extracted spectrum, and 

a step of performing a calculation operation on the 
20 estimated flicker component and the input image signal to 
cancel out the estimated flicker *component.= 

A flicker reduction method of a second invention for 
reducing a fluorescent light flicker component in each of 
color signals of colors obtained by photographing a subject 
25 through an XY addressing type image pickup element under an 
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illumination of a fluorescent lamp, includes 

a step of integrating the color signal of each color, 
as an input image signal, throughout a duration of time 
equal to or longer than one horizontal period, 

a step of normalizing the integrated value or a 
difference value between the integrated values of adjacent 
fields or adjacent frames, 

a step of extracting a spectrum of the normalized 
integrated value or the normalized difference value, 

a step of estimating a flicker component from the 
extracted spectrum, and 

a step of performing a calculation operation on the 
estimated flicker component and the input image signal to 
cancel out the estimated flicker component. 

A flicker reduction method of a third invention for 
reducing a fluorescent light flicker component in each of a 
luminance signal and each of color signals of colors, 
obtained by photographing a subject through an XY addressing 
type image pickup element under an illumination of a 
fluorescent lamp, includes 

a step of integrating each of the luminance signal and 
the color signal of each color, as an input image signal, 
throughout a duration of time equal to or longer than one 
horizontal period, 

a step of norinalizing the integrated value or a 
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difference value between the integrated values of adjacent 
fields or adjacent frames, 

a step of extracting a spectrum of the normalized 
integrated value or the normalized difference value, 
5 a step of estimating a flicker component from the 

extracted spectrum, and 

a step of performing a calculation operation on the 
estimated flicker component and the input image signal to 
cancel out the estimated flicker component. 

10 In accordance with the flicker reduction method of the 

present invention, a signal component other than a flicker 
component is removed as the normalized integrated value or 
the normalized difference value, and a signal that allows 
the flicker component to be easily estimated at a high 

15 precision is obtained from a black background portion and a 
low- illuminance portion, each portion having a small amount 
of flicker component. By extracting the spectrum to an 
appropriate order from the normalized integrated value and 
the normalized difference value, the flicker component is 

20 thus estimated at a high precision regardless of the type of 
a fluorescent lamp and a luminance varied waveform even in 
an area where a signal component becomes discontinuous by 
the effect of a subject. The flicker component is reliably 
and sufficiently reduced from the input image signal by 

25 performing the calculation operation on the estimated 
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flicker component and the input image signal . 

In particular, in accordance with the flicker component 
reduction method of the second or third invention, the 
flicker component is detected from the color signal of each 
5 color, obtained as the video signal, on a per color signal 
basis, or on a per luminance and color signal basis, and the 
detected flicker component is reduced. The fluorescent 
light flicker containing the color flicker is thus 
accurately detected and reliably and sufficiently reduced. 

10 

Brief Description \of the Drawings 

Fig. 1 illustrates a system configuration of an image 
pickup device of one embodiment of the present invention. 
Fig. 2 illustrates one example of a digital signal 
15 processor of a primary color system. 

Fig. 3 illustrates one example of a digital signal 
processor of a complementary color system. 

Fig. 4 illustrates a first example of a flicker ;reducer. 
Fig. 5 illustrates one example of a calculating block 
20 taking into a saturation region. 

Fig. 6 illustrates a second example of the flicker 
reducer. 

Fig. 7 illustrates a third example of the flicker 
reducer. 

25 Fig. 8 illustrates one example of the flicker reducer 
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under the illumination of a non- fluorescent lamp. 

Fig. 9 illustrates another example of the flicker 
reducer under the illumination of a non- fluorescent lamp. 

Fig. 10 illustrates an image pickup device that takes 
5 into account a subject that changes greatly in a short 

period of time in response to an operation or an action of a 
photographer. 

Fig. 11 illustrates one example of image pickup device 
that takes into account the flicker component reduction 
.10 process that becomes unnecessary depending on a 
photographing condition. 

Fig. 12 illustrates another example of image pickup 
device that takes into account the flicker component 
reduction process that becomes unnecessary depending on a 
15 photographing condition. 

Fig. 13 illustrates a basic configuration to adjust an 
estimated flicker component. 

Fig. 14 illustrates a first specific example for 
adjusting the estimated flicker component. 
20 Fig. 15 illustrates a second specific example for 

adjusting the estimated flicker component. 

Fig. 16 serves the purpose of explanation of Fig. 14 
and Fig. 15. 

Figs. 17A and 17B illustrate equations serving the 
25 purpose of explanation of an example of adjusting ^the 
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estimated flicker component. 

Figs. 18A and 18B illustrate equations serving the 
purpose of explanation of the example, of adjusting the 
estimated flicker component. 
5 Figs. 19A and 19B illustrate equations serving the 

purpose of explanation of the example of adjusting the 
estimated flicker component. 

Figs. 20A - 20E illustrate equations serving the 
purpose of explanation of the example of adjusting the 
10 estimated flicker component. 

Figs. 21A - 21C illustrate equations serving the 
purpose of explanation of the example of .adjusting the 
estimated flicker component . 

Figs. 22A and 22B serve the purpose of explanation of 
15 Fig. 8 and Fig. 9. 

Fig. 23 illustrates a subject used in tests. 

Fig. 24 is a plot of an integrated value of the subject 
of Fig. 23 . 

Fig. 25 is a plot of a difference value of the subject 
20 of Fig. 23 . 

Fig. 26 is a plot of a normalized difference value of 
the subject of Fig. 23. 

Fig. 27 is a plot of a flicker coefficient estimated • 
for the subject of Fig. 23. 
25 Fig. 28 illustrates a fluorescent light flicker of a 
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CCD image pickup device. 

Fig. 29 illustrates a fluorescent light flicker in an 
XY addressing type image pickup element. 

Fig. 3 0 illustrates a stripe pattern in one frame in 
5 the fluorescent light flicker in the XY addressing type 
image pickup element . 

Fig. 31 illustrates a stripe pattern in three 
consecutive frames in the fluorescent light flicker in the 
XY addressing type image pickup element. 

10 

Best Mode for Carrying Out the Invention 
[Embodiments of image pickup devices: Fig. 1 - Fig. 3] 
(System configuration: Fig. 1) 

Fig. 1 illustrates a system configuration of one 

15 embodiment of an image pickup device in accordance with the 
present invention. The image pickup device is here a video 
camera employing a CMOS image pickup element as a XY 
addressing type scanning image pickup element.- 

In the image pickup device of this embodiment, namely, 

20 the video camera, light from a subject is incident on a CMOS 
image pickup device 12 via an image pickup optical system 11, 
and photoelectrically converted into an analog video signal 
by the CMOS image pickup device 12. The resulting analog 
video signal is thus obtained from the CMOS image pickup 

25 device 12. 
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The CMOS image pickup device 12 includes, on a CMOS 
substrate, a plurality of two-dimensionally arranged pixels, 
each pixel including a photodiode (a photo gate) , a transfer 
gate (shutter transistor) , a switching transistor (address 
5 transistor) , an amplification transistor, a reset transistor 
(reset gate), etc., and further a vertical scanning circuit, 
a horizontal scanning circuit, and a video signal output 
circuit. 

The CMOS image pickup device 12 may be a primary color 

10 system or a complementary color system, and the analog video 
signal output from the CMOS image pickup device 12 is an RGB 
primary color signal or a complementary color signal. 

An analog signal processor 13 packaged in an IC 
(integrated circuit) processes the analog video signal from 

15 the CMOS image pickup device 12 on a per color signal basis, 
thereby sample-holding the analog video signal, gain 
controlling the analog video signal in an AGC (automatic 
gain control) process, and A/D converting the video signal 
into a digital signal . , , . . 

20 The digital video signal from the analog signal 

processor 13 is processed by a digital signal processor 20 
packaged in an IC, as will be discussed later. In the 
digital signal processor 20, a flicker reducer 25 reduces 
the digital video signal on a per signal component basis 

25 using a method of the present invention as will be discussed 
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later. The digital video signal is finally converted into a 
luminance signal Y, and red and blue color difference 
signals R-Y and B-Y to be output from the digital signal 
processor 20. 

5 A system controller 14, including a microcomputer, 

controls each block in the camera. 

More specifically, a lens drive control signal is 
supplied from the system controller 14 to a lens driver 15 
packaged in an IC, and the lens driver 15 drives lenses in - 

10 an image pickup optical system 11. 

The system controller 14 supplies a timing control 
signal to a timing generator 16. The timing generator 16 
supplies a variety of timing signals to. a CMOS image pickup 
element 12 to drive the CMOS image pickup element 12 . 

15 The system controller 14 receives detection signals for 

signal components from the digital signal processor 20. The 
analog signal processor 13 gain controls the color signal in 
response to an AGC signal from the system controller 14. 
Signal processing of the digital signal processor 20 is 

20 controlled by the system controller 14. 

If a subject changes in a short period of time in 
response to an action of a photographer, the system 
controller 14 detects an output from a hand- shake sensor 17 
connected to the system controller 14, and controls a 

25 flicker reducer 25 as will be discussed later. 
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A control panel 18a and a display 18b, forming an 
interface 18, are connected to the system controller 14 via 
an interface (I/F) 19 composed of a microcomputer.. The 
system controller 14 detects a setting operation and a 
5 selection operation on the control panel 18a. The system 
controller 14 also displays a set status and a control 
statue of the camera on the display 18b. 

If a subject changes greatly in a short period of time 
in response to a camera operation, such as a zoom operation, 
10 the system controller 14 detects the camera operation, and 
controls the flicker reducer 25 as will be discussed later. 

If no flicker reduction process is required, the system 
controller 14 detects that no flicker reduction is required 
and controls the flicker reducer 25 as will be discussed 
15 later. 

(Primary color system: Fig. 2) 

Fig. 2 illustrates an example of the digital signal 
processor 20 of a primary color system. 

The primary color system is a three -panel system or a 
20 single panel system. The three-panel system includes the 
image pickup optical system 11 of Fig. 1 including a light 
separation system for separating light from a subject into 
the RGB primary colors, and a CMOS image pickup element as 
the CMOS image pickup element 12 for the RGB colors. The 
25 single-panel system includes, as the CMOS image pickup 
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element 12, a CMOS image pickup element including RGB color 
filers periodically arranged on each pixel in a horizontal 
direction on the screen of a light input surface thereof. 
In this case, the RGB primary color signals are read from 
5 the CMOS image pickup element 12 . 

In the digital signal processor 20 of Fig. 1, a clamp 
circuit 21 clamps a black level of each .of the RGB primary 
color signals to a predetermined level, a gain adjustment 
circuit 22 gain adjusts the clamped RGB primary color 

10 signals in accordance with an amount of exposure, and 

flicker reducers 25R, 25G, and 25B reduce flicker components 
in the gain adjusted RGB primary color signals in accordance 
with a method of the present invention. 

In the digital signal processor 20 of Fig. 2, a white 

15 balance adjustment circuit 27 adjusts the flicker-reduced 
RGB primary color signals, a gamma correction circuit 28 
corrects the white-balanced adjusted RGB primary^ color 
signals, and a synthesis matrix circuit 29 generates the 
luminance signal Y, and the color difference signals R-Y and 

20 B-Y from the gamma -corrected RGB primary color signals. 

The primary color system generates the luminance signal 
Y after the processing of the RGB primary color signals is 
complete as shown in Fig. 2. Since the flicker component in 
the RGB primary color signals is reduced in the course of 

25 the process of the RGB primary color signals, the flicker 
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component in each of the color components and the luminance 
signal is sufficiently reduced. 

Although flicker reducers 25R, 25G, and 25B are 
preferably arranged as shown in Fig. 2, the present 
5 invention is not limited to this arrangement. 
(Complementary color system: Fig. 3) 

Fig. 3 illustrates one example of the digital signal 
processor 20 for a complementary color system. 

The primary color system is a single -panel system that 

10 includes, as the CMOS image pickup element 12 of Fig. 1, a 

CMOS image pickup element having complementary color filters 
on a light input surface thereof. The complementary color 
filter, as shown as a color filter 1 in Fig. 3, a green 
color filter section IG and a magenta color filter section 

15 IMg are alternately arranged on the pixels in a horizontal 
direction at one horizontal line position Lo at every two 
lines, and a cyan color filter section ICy and a yellow 
color filter lYe are alternately arranged on the pixels in a 
horizontal direction at the other horizontal line position 

20 Le at every two lines. 

In this case, the CMOS image pickup element 12 
synthesizes and reads video signals of two adjacent 
horizontal line positions. At each horizontal period, a 
color signal synthesized from the green color signal and the 

25 cyan color signal and a color signal synthesized from the 
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magenta color signal and the yellow color signal are 
alternately obtained at every pixel clock from the CMOS 
image pickup element 12 • 

In the digital signal processor 20 of Fig. 3, a clamp 
5 circuit 21 clamps a black level of the complementary color 
signal to a predetermined level, a gain adjustment circuit 
22 gain adjusts the clamped complementary color signal in 
response to an amount of exposure, a luminance synthesizing 
circuit 23 generates a luminance signal Y from the gain- 
10 adjusted complementary color signal, and a primary color 
separating circuit 24 separates the gain-adjusted 
complementary color signal into the RGB primary color 
signals. 

In the digital signal processor 20 of Fig. 3, a flicker 
15 reducer 25Y reduces a flicker component in the luminance 

signal Y from the luminance synthesizing circuit 23, and the 
flicker reducers 25R, 25G, and 25B reduce flicker components 
in the RGB primary color signals from the primary color 
separating circuit 24 in accordance with a method of the ' 
20 present invention. . 

In the digital signal processor 20 of Fig. 3, a gamma 
correction circuit 26 corrects the gradation of the flicker- 
reduced luminance signal, thereby resulting in a luminance 
signal Y as an output, a white balance adjustment circuit 27 
25 white-balance adjusts the flicker-reduced RGB primary color 
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signals, a gamma correction circuit 28 adjusts the gradation 
of the white-balance adjusted RGB primary color signals, and 
a synthesis matrix circuit 29 generates the color difference 
signals R-Y and B-Y from the gamma -corrected RGB primary 
5 color signals. 

In the primary color system, the luminance signal and 
the RGB primary color. signals are generated in a relatively 
earlier phase of the process of the digital signal processor 
20 as shovm in Fig. 3. This is because the luminance signal 

10 is easily generated from the synthesized signal through a 
simple additive process. If the RGB primary color signals 
are generated from the synthesized signal through a 
difference process, and the luminance signal Y is generated 
from the RGB primary color signals, and an S/N ratio of the 

15 luminance signal is lowered. 

If the luminance signal Y and the color signals are 
processed through separate lines, merely reducing the 
flicker component in each color signal as not sufficient in 
the reduction of the flicker component in the luminance 

20 signal. The flicker components in each color component and 
the luminance signal are sufficiently reduced by separately 
reducing the flicker component in the luminance signal as 
shown in Fig. 3. 

The flicker reducers 25R, 25G, and 25B are preferably 

25 arranged as shown in Fig. 3, but the present invention is 
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not limited to this arrangement. 

[Embodiments of the flicker reduction method: Fig. 4 - 
•Fig. 27] 

The flicker reducers 25R, 25G, and 25B of Fig. 2, and 
5 the flicker reducers 25R, 2SG, and 25B of Fig. 3 are 

constructed as discussed below. The flicker reducers 25R, 
25G, and 25B are hereinafter collectively referred to as a 
flicker reducer 25. 

In the following discussion, an input image signal 

10 refers to a RGB primary color signal or a luminance signal 

input to the flicker reducer 25 prior to a flicker reduction 
process, and an output image signal refers to a RGB primary 
color signal or a luminance signal, output from the flicker 
reducer 25, subsequent to the flicker reduction process . 

15 In the following discussion, a subject is photographed 

by a CMOS camera of an NTSC system (with a vertical 
•synchronization frequency of 60 Hz) under an illumination of 
a fluorescent lamp operating from a commercial AC power 
source of 50 Hz. When the fluorescent light flicker is not 

20 reduced, the brightness level variations and the color 
variations take place not only between fields but also 
within each field and a strip pattern of five periods (five 
wavelengths) appears over three fields (three wavelengths) 
on a screen as shown in Figs. 29-31. 

25 In a non- inverter fluorescent lamp as well as an 
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inverter fluorescent lamp, flickering takes place if 
rectification is not sufficient. The present invention is 
thus not limited to the non-inverter type. 

(First example of the flicker reduction method: Fig. 4) 
5 Fig. 4 illustrates a first example of the flicker 

reducer 25. 

As shown in Figs. 30 and 31, the subject is uniform. 
The flicker component is typically proportional to a signal 
intensity of the subject. 

10 Let In' (x,y) represent an input image signal (an RGB 

primary color signal or a luminance signal prior to the 
flicker reduction process) of a subject at any pixel n(x,y) 
in any field n, and In' {x,y) is a sum of a signal component 
free from the flicker component and the flicker component 

15 proportional to the signal component as represented in 
equation (1) in Fig. 17A. 

In(x,y) represents the signal component, Fn (y) *In (x, y) 
represents the flicker component, and Tn (y) represents a 
flicker coefficient. One horizontal period is sufficiently 

20 shorter than a light emission period (l/lOO second) of a 

fluorescent lamp, and the flicker coefficient is considered 
as being constant on the same line in the same field, and 
the flicker coefficient is thus represented by FnCy) . 

As represented by equation (2) in Fig. 17A, rn(y) is 

25 expressed by Fourier series for generalization. In this way. 
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the flicker coefficient is expressed in a form that accounts 
for all of emission characteristics and persistence 
characteristics, different from type to type of fluorescent 
lamp . 

5 In equation (2) , Xo represents the wavelength of the on- 

screen flicker shown in Fig. 30. Let M represent the number 
of read lines per field, and Xo is L (=M*60/100) lines. 
Here, (Do represents a normalization angular frequency 
normalized by A.o. 

10 Here, ym represents an amplitude of the flicker 

component of each order (m=l, 2, 3, . . .) . <I>mn represents an 
initial phase of the flicker component of each order, and is 
determined by the light emission period (1/100 second) and 
the exposure timing of the fluorescent lamp. Since Omn 

15 reverts back to the same value every three fields, and a 
difference of Omn to the immediately preceding field is 
determined by equation (3) in Fig. 17A. 

<Calculation and storage of integrated value> 

In the example of Fig. 4, an integrating block 31 

20 integrates the input image signal In'(x,y) over one line in 
a horizontal direction on the screen as shown in equation 
(4) of Fig. 17B to minimize the effect of a picture in the 
detection of a flicker. An integrated value Fn(y) is thus 
calculated- As represented by equation (5) of Fig. 17B, 

25 an(y) in equation (4) represents a value that is obtained by 
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integrating a signal component In(x,y) over one line. 

The integrated value Fn (y) thus calculated is stored in 
an integrated value storage block 32 to detect flickering in 
subsequent fields. The integrated value storage block 32 is 
5 designed to store the integrated valueis of at least two 
fields. 

If a subject is uniform, the integrated value an(y) of 
the signal component In{x,y) becomes constant. It is thus 
easy to extract a flicker component an(y)*rn(y) from the 

10 integrated value Fn(y) of the input image signal In' (x,y) . 

In a general subject, however, an(y) contains m*(oo 
component, and it is impossible to separate a luminance 
signal and a color component, as a flicker component, from a 
luminance signal and a color component as a signal component 

15 of the subject itself. It is impossible to extract only the 
flicker component. The flicker component in a second term 
is significantly smaller than the signal component in a 
first term in equation (4) , and the flicker component is 
almost buried in the signal component. 

20 Fig. 24 illustrates the integrated value Fn(y) in three 

consecutive fields of a subject (a color subject in 
practice) of Fig. 23 for reference. Fig. 24 is a plot of 
the integrated value Fn(y) of a red color, and Field: N+0 
(solid line). Field: N+1 (dash line), and Field: N+2 (broken 

25 line) represent first, second, and third fields appearing 
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consecutively. As seen from Fig. 24, it is impossible to 
extract the flicker component directly from the integrated 
value Fn (y) . 

<Average value calculation and difference calculation> 
5 As shown in Fig. A, the integrated values of the three 

consecutive fields is used to remove the effect of an(y) 
from the integrated value Fn (y) . 

In this example, during the calculation of the 
integrated value Fn(y), an integrated value Fn__l (y) on the 

10 same line in the first preceding field immediately prior to 
the current field and an integrated value Fn_2 (y) on the 
same line in the second preceding field immediately prior to 
the first preceding field are read from the integrated value 
storage block 32. A average value calculating block 33 

15 calculates an average value AVE[Fn{y)] of the three 
integrated value Fn (y) , Fn__l (y) , and Fn_2 (y) . 

If the subject is identified as the same entity during 
the three consecutive fields, an(y) may be assumed to have 
•the same value. If the motion of the subject is ismall 

20 enough during the three consecutive fields, this assumption 
presents no problem in practice. In the calculation of the 
average of the integrated value during the three consecutive 
fields, equation (3) means that signals with the phases of 
the flicker components thereof successively shifted by (- 

25 27i/3)*m are summed. As a result, the flicker components 
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cancel out each other. The average value AVE [Fn (y) ] is 
expressed by equation (6) of Fig. 18A. 

The average value of the integrated values for the 
three consecutive fields is calculated based on the 
5 assumption that the approximation of equation (7) of Fig. 
18A holds. If the motion of the subject is large, the 
approximation of equation (7) does not hold. 

In the flicker reducer 25 intended to work with a 
subject moving in a large amount, the integrated value 

10 storage block 32 stores the integrated values over at least 
three fields, and the average value of the integrated values 
of at least four fields including the integrated value Fn (y) 
of the current field. In this way, the operation of low- 
pass filters functioning in time axis reduces the effect of 

15 the moving subject. 

However, the flicker is repeated every three fields. 
To cancel out the flicker component, the average value of 
the integrated values of j (an integer multiple of 3, equal 
to or greater than twice 3, namely, 6, 9, .-.) fields needs 

20 to be calculated. The integrated value storage block 32 is 
thus designed to store the integrated valueis of at least (j- 
1) fields. 

Fig. 4 illustrates a case in which the approximation of 
equation (7) of Fig. 18A holds. Furthermore in this example, 
25 a difference calculating block 34 calculates a difference 
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between the integrated value Fn(y) of the current field from 
the integrating block 31 and the integrated value Fn_l (y) of 
the field immediately prior to the current field from the 
integrated value storage block 32. The difference value 
5 Fn (y) -Fn_l (y) represented by equation ■ (8) of Fig. 18B is 

thus calculated. Equation (8) also holds on condition that 
the approximation of equation (7) holds. 

Fig. 25 illustrates the difference value Fn (y) -Fn_l (y) 
of the three consecutive fields of the subject of Fig. 23. 

10 Since the effect of the subject is sufficiently removed from 
the difference value Fn (y) -Fn_l (y) , the state of the flicker 
component (flicker coefficient) more distinctly appears than 
in the integrated value Fn(y) of Fig. 24. 
<Normalization of difference value> 

15 Furthermore in the example of Fig. 4, the normalizing 

block 35 normalizes the difference value Fn (y) -Fn_l (y) from 
the difference calculating block 34 by dividing the 
difference value Fn (y) -Fn_l (y) by the average value 
AVE[Fn(y)] from the average value calculating .block 33, 

20 thereby outputting the normalized difference value gn{y) . 

The normalized difference value gn(y) is expanded as 
represented by equation (9) of Fig. 19A using equation (6) 
of Fig. 18A and equation (8) of Fig. 18B and the sum-to- 
product formula of trigonometry. The normalized difference . 

25 value gn(y) is also expressed by equation (10) of Fig. 19B 
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using the relationship of equation (3) of Fig. 17A. Here, 
.[Am I and 9m in equation (10) are respectively represented by 
equations (11a) and (lib) . 

Since the effect of the signal intensity of the subject 
5 still persists in the difference value Fn (y) -Fn_l (y) , the 
levels of luminance variations and color variations due to 
the flickering are different from area to area. By 
normalizing the difference value gn(y) as described above, 
levels of the luminance variations and the color variations 
10 due to the flickering are set to be consistent to the same 
level • 

Fig. 26 illustrates the normalized difference value 
gn(y) of the three consecutive fields of the subject of Fig. 
23 . 

15 <Estimation of the flicker component through spectrum 

extraction> 

I Am I and 0m respectively represented by equations (11a) 
and (lib) of Fig. 19B are an amplitude and an initial phase 
of the spectrum of each order of the normalized difference 

20 value gn(y) . The normalized difference value gn(y) is 

Fourier transformed to determine the amplitude |Am| and the 
initial phase 0m of the spectrum of each order. Equations 
(12a) and (12b) of Fig. 20A show that an amplitude ym and an 
initial phase Omn of the flicker component of each order 

25 represented by equation (2) of Fig. 17A are thus obtained. 
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In the example of Fig. 4, a DFT block 51 discrete 
Fourier transforms data of one wavelength (of L lines) of 
flickering of the normalized difference value gn(y) from the 
normalizing block 35. 
5 Let DFT[gn(y)] represent a DFT operation, and Gn(m) 

represent the DFT result of order m, and the DFT operation 
is expressed by equation (13) of Fig. 20B. Here, W in 
equation (13) is represented by equation (14) . According to 
the definition of tthe DFT, the relationship between 
10 equations (11a) and (lib) and equation (13) is represented 
by equations (15a) and (15b) of Fig. 20C. 

The amplitude ym and the initial phase <Dmn of the 
flicker of each order are thus determined from equations 
(12a) , (12b) , (15a) , and (15b) with reference to equations 
15 (16a) and (16b) of Fig. 20D. 

The data length of the DFT operation is set to the one 
wavelength (of L lines) of the flicker because a discrete 
spectrum group of an integer multiple of oo can be directly 
obtained. 

20 The FFT (Fast Fourier Transform) is typically used as a 

Fourier transform for digital signal processing. However, 
the DFT is intentionally used in this embodiment of the 
present invention. The DFT is more convenient than the FFT 
because the data length of the Fourier transform is not a 

25 power of 2. Alternatively, the FFT can also be used by 
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manipulating input data and output data. 

The flicker component is sufficiently approximated 

under the illumination of an actual fluorescent lamp even if 

the order number is limited to m-th order. It is not 
5 necessary to output all data in the DFT operation. In 

comparison with the FFT, the DFT suffers from any particular 

drawback in terms of operation efficiency in the application 

of this invention. 

The DFT block 51 extracts the spectrum through the DFT 
10 operation defined by equation (13) , and then estimates the 

amplitude ym and the initial phase C>mn of the flicker 

component of each order through an operation represented by 

equations (16a) and (16b) . 

In the example of Fig. 4, a flicker generating block 53 
15 calculates a flicker coefficient Fn (y) represented by 

equation (2) of Fig. 17A from the estimated values ym and 

Omn from the DFT block 51. 

As previously discussed, the flicker component is 

sufficiently approximated under the illumination of the 
20 light of the fluorescent lamp even if the order number is 

limited to the m-th order. In the calculation of the 

flicker coefficient rn(y) through equation (2) , the order of 

total sum is set to a predetermined order/ such as a second 

order, rather than infinity. 
25 Fig. 27 illustrates the flicker coefficient FnCy) of 
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the three consecutive fields of the subject of Fig. 23. 

In the preceding method, the difference value Fn(y)- 
Fn_l (y) is calculated and then normalized 'by the average 
value AVE [Fn (y) ] in a black background portion and a low- 
5 illuminance portion, where the flicker component is 
typically small and is entirely buried in the signal 
component in the integrated value Fn (y) . The flicker 
component is thus accurately detected. 

The estimation of the flicker component from the 
10 spectrum to an appropriate order means that the 

approximation is effected with the noirmalized difference 
value gn(y) incompletely reproduced. Even if a 
discontinuity takes place in the normalized difference value 
gn(y), as a result of an incomplete reproduction, depending 
15 on the state of the subject, the flicker component of that 
portion is accurately estimated . 

<Calculation for flicker reduction> 

From equation (1) of Fig. 17A, the signal component 
In(x,y) having no flicker component is expressed in equation 
20 (17) of Fig. 20E. 

In the example of Fig. 4, a calculating block 40 adds 1 
to the flicker coefficient rn(y) from the flicker generating 
block 53, and the input image signal In* (x,y) is divided by 
the resulting sum :[l+rn(y)] . 
25 The flicker component contained in the input image 
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signal In' (x,y) is almost entirely removed in this way. The 
calculating block 40 results in a signal component In(x,y) 
having no substantial flicker component as an output image 
signal (as the RGB primary color signal or the luminance 
5 signal subsequent to flicker reduction) . 

If all above processes are not completed within the 
duration of time of one field due to constraints on 
calculation performance of the system, a function of storing 
the flicker coefficient rn(y) over the three fields is 

10 provided in the calculating block 40 taking advantage of the 
repetition of the flickering every three fields. The stored 
flicker coefficient Tn (y) is calculated for the input image 
signal In' (x,y) subsequent to the three fields. 

(Example accounting for saturation region: Fig.. 5) 

15 If the calculating block 4 0 of Fig. 4 performs the 

calculation represented by equation (17) with the level of 
the input image signal In* (x,y) falling within a saturation 
region, the signal component (a color component or a 
luminance component) varies against the object of the 

20 invention. For this reason, the calculating block 40 
preferably has a structure of Fig. 5. i 

The calculating block 40 of Fig. 5 includes an adder 
circuit 41 for adding 1 to the flicker coefficient rn(y) 
from the flicker generating block 53, a divider circuit 42 

25 for dividing the input image signal In' (x,y) by the sum 
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[l+rn(y)], a switch 43 on the input side, a switch 44 on the 
output side, and a saturation level determining circuit 45. 
The saturation level determining circuit 45 determines on a 
per pixel basis whether the level of the input image signal 
5 In* (x,y) is not lower than a threshold level of the 
saturation region. 

If it is determined that the level of the input image 
signal In' (x,y) is lower than the threshold level of the 
saturation region on each pixel, the saturation level 
10 determining circuit 45 sets the switches 43 and 44 to the 
sides thereof to the divider circuit 42. As previously 
discussed, the result of the calculation of equation (17) is 
output from the calculating block 4 0 as an output image 
signal. 

15 If it is determined that the level of the input image 

signal In' (x,y) is not lower than the threshold level of the 
saturation region on the pixel, the saturation level 
determining circuit 45 sets the switches 43 and 44 to the 
sides thereof opposite from the divider circuit 42. The 

20 input image signal In' (x,y) is output as is as an output 
signal from the calculating block 40. 

When the level of the input image signal In* (x,y) falls 
within the saturation region, the signal component (the 
color component or the luminance component) is free from 

25 variations, and a high-quality image signal results. 
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(Second example of the flicker reduction method: Fig. 

6) 

If the difference value Fn (y) -Fn_l (y) is normalized by 
the average value AVE[Fn(y)] as shown in Fig. 4, a finite 
5 calculation accuracy is effectively achieved. = If 
calculation accuracy requirement is satisfied, the 
integrated value Fn(y) can be directly normalized by the 
average value AVE [Fn (y) ] . 

Fig. 6 illustrates such an example. The normalizing 
10 block 35 perfoinns a normalization operation by dividing the 
integrated value Fn(y) from the integrating block 31 by the 
average value AVE[Fn(y)] from the average value calculating 
block 33, thereby putputting the normalized difference value 
gn (y) . 

15 The normalized difference value gn (y) is represented by 

equation (18) of Fig. 21A. As represented by equation (19) 
of Fig. 21B, a subtracter circuit 36 subtracts 1' from the 
normalized difference value gn(y) represented by equation 
(18) and transfers the subtraction results ^to the DFT 

20 (discrete Fourier transform) block 51 to make the subsequent 
process of the arrangement of Fig. 6 identical 'to * the 
subsequent process of the arrangement of. Fig. 4. : 

Since |Am|=ym and 0m=<I>mn, ym and Omn are determined from 
equations (20a) and (20b) with reference to equations (15a) 

25 and (15b) of Fig. 20C. 
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In the example of Fig. 4, the DFT block 51 estimates 
the amplitude ym and the initial phase Omn of the flicker 
component of each order through the calculation of equations 
(16a) and (16b) after extracting the spectrum through the 
5 DFT operation defined by equation (13) . In the example of 
Fig. 6, the DFT block 51 estimates the amplitude ym and the 
initial phase Omn of the flicker component of each order 
through the calculation of equations (20a) and (20b) after 
extracting the spectrum through the DFT operation defined by 
10 equation (13) . The subsequent process remains unchanged 
from that of the arrangement of Fig. 4. 

Since the example of Fig. 6 does not need the 
difference calculating block 34, the flicker reducer 25 is 
simplified accordingly. 
15 In this example as well, the calculating block 40 

preferably has the structure of Fig. 5. 

(Third example of the flicker reduction method: Fig. 7) 
If the approximation defined by equation (7) of Fig. 
18A holds, the average value AVE[Fn(y)] for use in the 
20 normalization in the example of Fig. 4 equals an(y) defined 
by equation (6), and the second term [an (y) *rn (y) ] of 
equation (4) of Fig. 17B significantly smaller than the 
first term an (y) has an insignificant effect on the 
normalization. 

25 The integrated value Fn(y) is used instead of the 
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average value AVE[Fn(y)] in the normalization without any- 
particular problem. The flicker component is effectively 
detected in the same manner as when the average value 
AVE[Fn(y)] is used. 
5 In the example of Fig. 7, the normalizing block 35 

performs a normalization operation by dividing the 
difference value Fn (y) -Fn_l (y) from the difference 
calculating block 34 by the integrated value Fn (y) from the 
integrating block 31. The subsequent process remains 

10 unchanged from that of the arrangement of Fig. 4. 

In the example of Fig. 7, it is sufficient if the 
integrated value storage block 3 2 holds the integrated value 
of one field, and the average value calculating block 33 is 
not required. The flicker reducer 25 is thus simplified in 

15 structure . 

In this example as well, the calculating block 4 0 
preferably has the structure of Fig. 5. 

(Example of another process performed under non- 
fluorescent lamp: Fig. 8 and Fig. 9) 

20 No particular problem is presented in the above- 

referenced flicker reduction process when a photographing 
operation is performed under the illumination of a non- 
fluorescent lamp (under an non- fluorescent illumination 
environment) . However, if an otherwise unnecessary process 

25 is performed, the effect on image cjuality becomes a concern 
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even if the flicker component is low enough. 

If a photographing operation is performed under the 
illumination of the non- fluorescent lamp, the flicker 
reduction process is preferably disabled. The flicker 
5 reducer 2 5 is preferably designed to output the input image 
signal In* (x,y) as is as an output image signal from the 
flicker reducer 25. 

Fig. 8 illustrates an example of such a flicker reducer 
25. A normalized and integrated value calculating block 30 

10 is designed as shown in the example of Fig. 4, Fig. 6 or Fig 
7. In the example of Fig. 4 and Fig. 1, the difference 
value Fn (y) -Fn_l (y) rather than the integrated value Fn(y) 
is normalized, and for convenience of explanation, the 
corresponding block is also referred to as a normalized and 

15 integrated value calculating block. 

In the example of Fig. 8, a under- fluorescent -lamp 
determination block 52 is arranged between the DFT block 51 
and the flicker generating block 53. 

Under the illumination of the fluorescent lamp, the 

20 level (amplitude) ym of the calculated component of each 

order is sufficiently higher at m=l than a threshold Th and 
sharply lowers as m increases as shown in Fig. 22A. Under 
the illumination of the non- fluorescent lamp, the level of 
the component is lower than the threshold Th at every order 

25 as shown in Fig. 22B. 
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The spectrum is ideally zero under the illumination of 
the non- fluorescent lamp. In practice, however, the subject 
moves, and the normalized difference value. gn(y) or the 
normalized integrated value gn{y)-l, generated from the 
5 signals of a plurality of consecutive fields, inevitably 
contains a tiny amount of frequency component. 

The under- fluorescent -lamp determination block 52 thus 
determines whether the level at m=l is above the threshold 
Th. If it is determined that the level at m=l is above the 

10 threshold Th, the under- fluorescent -lamp determination block 
52 determines that the photographing operation is performed 
under the illumination of the fluorescent lamp. -The 
estimated values of ym and Omn from the DFT block 51 are 
directly output to the flicker generating block 53. In this 

15 case, the above -described flicker reduction process is 
executed. 

If the level of the component at m=l is lower than the 
threshold Th, the under- fluorescent -lamp determination block 
52 determines that the photographing operation is performed 
20 under the non- fluorescent lamp, and sets the -estimated value 
of ym of all order m to be zero. The flicker coefficient 
rn(y) becomes zero, and the input image signal In' (x,y) is 
output as is as an output image signal from the calculating 
block 40. 

25 Fig. 9 illustrates such an example. In this example. 
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as in the example of Fig. 8, the under- fluorescent- lamp 
determination block 52 determines whether the photographing 
operation is performed under the under the illumination of 
the fluorescent lamp. If it is determined that the 
5 photographing operation is performed under the illumination 
of the non- fluorescent lamp, the under- fluorescent -lamp 
determination block 52 sets a detection flag COMP_OFF, stops 
the process of the flicker generating block 53 and the » 
calculating block 40, and outputs the input image signal 

10 In* (x,y) as is as an output image signal from the 

calculating block 40. If it is determined that the 
photographing operation is performed under the illumination 
of the fluorescent lamp, the under- fluorescent -lamp 
determination block 52 resets the detection flag COMP_OFF 

15 and the flicker reduction operation is thus performed. 

If the photographing operation is performed under the 
illumination of the non- fluorescent lamp in the example of 
Fig, 9, not. only an adverse effect on image quality is 
prevented but also power consumption is reduced.* 

20 (Subject greatly changing in response to an operation 

or an action of a photographer: Fig. 10) 

A subject occasionally changes in a short period of 
time in response to zooming, panning, tilting, or hand- 
shaking of a photographer. In such a case, the assumption 

25 of equation (7) of Fig. 18A fails to hold. As a result, the 
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flicker detection accuracy is degraded. 

The image pickup device is constructed as =shown in Fig. 
10 taking into consideration a subject that changes greatly 
in a short period of time in response to an operation or an 
5 action of a photographer. 

As shown in Fig. 10, switches 55 and 56 and a flicker 
storage block 57 are arranged between the flicker generating 
block 53 and the calculating block 40 additionally to the 
flicker reducer 25 of Fig. 4, Fig. 6, or Fig. 7. The 
10 detection flag DET_OFF, to be discussed later, output from 

the system controller 14 is fed to the switches 55 and 56 as 
a switching signal . ' 

The flicker storage block 57 stores the flicker 
coefficient rn(y) of the three fields. Each time the 
15 process of one field is complete, the flicker coefficient 

rn(y) is stored for use in the next field- The output read 
from the flicker storage block 57 is switched every three 
fields . 

The system controller 14 detects a subject if the 
20 subject changes greatly in a short period of time in 

response to the operation or the action of the photographer- 
For example, if the photographer presses a zoom key on 
the control panel 18a, the system controller^ 14 detects the 
pressing via the interface 19. The system controller 14 
25 controls lenses in response to the photographer ' s zoom 
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operation such as a telephoto and a wide-angle operation. 
The hand-shaking of the photographer is detected by the 
hand-shake sensor 17, and the information of the hand- 
shaking is sent to the system controller 14 . In response to 
5 the hand-shake information, the system controller 14 

performs anti-shake control. Upon detecting panning or 
tilting, the system controller 14 lightens the degree of 
anti-shake correction during panning, for example. Such 
control techniques remain unchanged from those that are 

10 performed in ordinary cameras - 

In the example of Fig. 10, the system controller 14 
sets the detection flag DET_OFF if the operation or the 
action of the photographer causing the subject to be changed 
greatly in a short period of time is detected. The system 

15 controller 14 resets the detection flag DET_OFF if neither 
of such operation and action is performed. 

In a normal state under which the subject does not 
change greatly in a short period of time, the detection flag 
DET_OFF is reset. In the flicker reducer 25, 'the switch 55 

20 is set to the side thereof to the flicker generating block 
53- The flicker coefficient FnCy) is fed from the flicker 
generating block 53 to the calculating block 40. The 
flicker reduction operation is performed. With the switch 
56 turned on, the flicker coefficient rn(y) is stored in the 

25 flicker storage block 57. 
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The detection flag DET_OFF is set if the subject 
changes greatly in a short period of time in response to the 
operation or the action of the photographer. In the flicker 
reducer 25, the switch 55 is set to the side thereof to the 
5 flicker storage block 57. The calculating block 40 receives 
the flicker coefficient rn(y) at a high detection precision 
level immediately prior to the operation or the action of 
the photographer, read from the flicker storage block 31, 
instead of the flicker coefficient Fn (y) at a low detection 

10 precision level. The flicker reduction operation is 

performed. The switch 56 is turned off, and thus prevents 
the flicker coefficient Fn (y) at the low detection precision 
level from being stored onto the flicker storage block 57. 
The flicker detection accuracy is heightened even when 

15 the subject changes greatly in a, short period of time in 

response to the operation or the action of the photographer. 
The flicker is reliably and sufficiently reduced. 

Furthermore, the detection flag DET_OFF is supplied to ■ 
the normalized and integrated value calculating block 30, 

20 the DFT block 51, and the flicker generating block 53. If 
the subject changes greatly in a short period of time in 
response to the operation or the action of the photographer, 
the detection flag DET_OFF is set. The set detection flag 
DET_OFF stops the process of each of the normalized and 

25 integrated value calculating block 30, the DFT block 51, and 
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the flicker generating block 53. In this case, power 
consumption is also reduced. 

In this case, the flicker coefficient Fn (y) is replaced 
with the immediately prior flicker coefficient thereof. ,A 
5 more front stage signal, for example, an integrated value 
Fn(y) , may be replaced with the immediately prior signal 
thereof . 

(Example of another process performed depending on 
photographing condition: Fig. 11 and Fig. 12) 

10 As will be discussed later, the flicker reduction 

operation becomes unnecessary depending on the photographing 
condition. In view of the effect on image quality, 
performing otherwise unnecessary flicker reduction operation 
is undesirable when the photographing operation is performed 

15 under the illumination of the non- fluorescent lamp as 
described above . 

First, the photographing conditions that require no 
flicker reduction operation include the case in which a 
video camera or a digital still camera, each capable of 

20 taking both a moving image and a still image, photographs a 
still image. 

A camera employing an XY addressing type image pickup 
element such as a CMOS image pickup element can set exposure 
timings of all pixels in one frame (including exposure start 
25 timings and exposure end timings) to be uniform, and can 
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control the effect of the fluorescent light flicker. A 
reading operation from the image pickup element is free from 
the limitation of frame rate that is applied to moving image 
capturing, and is thus performed at a slow speed in a light 
5 blocked state with a mechanical shutter closed. 

Based on a camera operation on the control panel 18a in 
the embodiment of Fig. 1, the system controller 14 
determines whether to take a still image with the exposure 
of all pixels in one frame set to the same timing. 

10 Second, the photographing states that need no flicker 

reduction operation also include the case in which a 
photographing operation is performed outdoors under the 
light of the sun, and the case in which the exposure time 
(electronic shutter time) is set to be an integer multiple 

15 of periods (l/lOO second) of luminance changes of the 
fluorescent light by adjusting the amount of exposure. 

Whether the photographing operation is performed under ♦ 
the illumination of the fluorescent lamp is ^determined by 
referencing the level of the spectrum ^extracted by the DFT 

20 block 51 as previously discussed with reference to Figs. 8 

and 9. The photographing operation performed outdoors under 
the light of the sun is categorized as a photographing 
operation under the illumination of the non- fluorescent lamp, 
and in this case, the system controller 14 directly 

25 determines from the amount of light of a subject that a 
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photographing operation is performed under the illumination 
of the non- fluorescent lamp. 

As previously discussed, the camera employing the XY 
addressing type image pickup element such as a CMOS image 
5 pickup element is free from the fluorescent light flicker if 
the exposure time is set to be an integer multiple of the 
periods (l/lOO second) of the luminance changes of the 
fluorescent lamp. The system controller 14 directly detects 
whether the exposure time is set to be an integer multiple 

10 of periods of the luminance change of the fluorescent lamp 
by adjusting the exposure amount. 

If the system controller 14 determines that the 
photographing state requires no flicker reduction operation, 
the flicker reduction operation is not effected- The system 

15 is designed so that the input image signal In'=(x,y) is 
output as is as an output image signal from the flicker 
reducer 25 . 

Fig. 11 illustrates one example of such a system 
configuration. In this example of the flicker reducer 25, a 
20 nulling block 59 is arranged between the DFT block 51 and 
the flicker generating block 53. The nulling block 59 is 
controlled by a flicker reduction on/off control signal from 
the system controller 14 . 

If the system controller 14 determines that the flicker 
25 reduction operation is required, the flicker reduction 
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on/off control signal is set to an on state, and the nulling 
block 59 outputs the estimated values of ym and Omn from the 
DFT block 51, as are, to the flicker generating block 53, 
In this case, the flicker reduction operation is effected. 
5 If the system controller 14 determines that the flicker 

reduction operation is unnecessary, the flicker reduction 
on/off control signal is set to an off state. The nulling 
block 59 nulls the estimated value of ym of the order m to 
zero. In this case, the flicker coefficient Tniy) also 
10 becomes zero, and the input image signal In' (x,y) is output 
as is as an output image signal from the calculating block 
40 . 

Fig. 12 illustrates still another example. The 
calculating block 40 in the flicker reducer 25 includes the 

15 adder circuit 41, the divider circuit 42 and the switches .43 
and 44, all shown in Fig. 5, but does not include the 
saturation level determining circuit 45 of Fig. 5. The 
switches 43 and 44 are controlled for switching by the 
flicker reduction on/off control signal from the system 

20 controller 14. 

If the system controller 14 determines that the flicker 
reduction operation is necessary, the switches 43 and 44 are 
set to the side thereof to the divider circuit 42. As 
previously discussed, the calculation result of equation 

25 (17) is output as an output image signal from the 



- 51 - 

S03P1324 

calculating block 40. 

If the system controller 14 determines that no flicker 
reduction operation is necessary, the switches 43 and 44 are 
set to the sides thereof opposite from the divider circuit 
5 42, and the input image signal In* (x,y) is output as is as 
an output image signal from the calculating block 40. 

In the example of Fig. 12, the flicker reduction on/off 
control signal is fed to the normalized and integrated value 
calculating block 30, the DFT block 51, and the flicker 
10 generating block 53. If the system controller 14 determines 
that no flicker reduction operation is necessary, the 
process of each of the normalized and integrated value 
calculating block 30, the DFT block 51 and the flicker 
generating block 53 is stopped. Power: consumption is thus 
15 reduced in this example. 

(Adjustment of detected flicker component: Fig. 13 - 
Fig. 15) 

In accordance with each of the above -referenced methods, 
the flicker component in the input image signal is reliably 
20 and effectively reduced when the fluorescent light flicker • 
is steadily and regularly generated. 

However, in each of the above -referenced methods, the 
average calculation or the difference calculation is 
performed throughout a plurality of fields to detect the 
25 flicker component. For this reason, the flicker component 
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cannot be accurately detected in a transitional and unstable 
state, for example, at the moment the fluorescent lamp is 
switched on or off or at the moment of any person's entrance 
into or exit out of the room illuminated by a fluorescent 
5 lamp. If the flicker reduction operation is effected in 
accordance with the flicker component obtained in such a 
transitional and unstable state, an undesirable correction 
can be performed on the input image signal . 

A variation in an angle of field in a horizontal 

10 direction can be caused in response to a horizontal motion 

of a subject, a camera operation such as panning and zooming, 
and hjand shaking of a photographer. Such variations are 
reliably and stably reduced. However, flicker reduction 
performance is slightly lowered for a variation in the angle 

15 of view in a vertical direction caused in response to a 

vertical motion of the subject, the camera operation such as 
panning and zooming, and hand shaking of the photographer. 

This problem is eliminated by the method illustrated in 
Fig. 10, In accordance with this method., however, the 

20 flicker component rn(y) is switched between when the subject 
is stable free from a large variation in a short period of 
time and when the subject changes greatly in a short period 
of time in response to an operation or an action ^of the 
photographer. For this reason, the photographer may feel 

25 the image of the subject discordant. 
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The effect of external disturbance cannot be controlled 
even in a normal stable state. In the method of Fig. 10, 
the image pickup device responds directly to the external 
disturbance because of the fast tracking feature thereof, 
5 and suffers from an error in the flicker reduction against 
the object of the flicker reduction operation. 

To reduce the effect of external disturbance, a 
filtering operation using an LPF (low-pass filter) is 
performed in the course of estimating the flicker component, 

10 and a time constant thereof is prolonged to slow the 
tracking feature in the flicker estimation. 

However, the setting of slow tracking feature also 
results in a slow tracking feature in the above -referenced 
transitional state. The flicker reduction operation cannot 

15 be performed quickly although the flicker reduction 

operation is required, for example, at the moment the 
fluorescent lamp is switched on or at the moment of any 
person's entrance into the room illuminated by the 
fluorescent lamp. Conversely, the flicker reduction 

20 operation is continuously performed although the flicker 

reduction operation is required no longer, for example, at 
the moment the fluorescent lamp is switched off or at the 
moment of any person' s exit out of the room illuminated by 
the fluorescent lamp. 

25 Instead of performing the calculation operation on the 
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detected flicker component and the input image signal, the 
amplitude and the phase of the detected . flicker component 
are adjusted as necessary, and then the adjusted flicker 
component and the input image signal are subjected to the 
5 calculation operation. In this way, the flicker reduction 
operation is flexibly and properly performed in a variety of 
cases. 

In the following example, parameters related to the 
flicker reduction operation, for example, the amplitude and 

10 the phase of the estimated flicker component are adjusted. 
Fig. 13 illustrates a basic structure of such an 
arrangement. In this example, the system controller 14 
captures data of the amplitude ym and the initial phase Omn 
of the estimated flicker component from the DFT block 51 in 

15 the flicker reducer 25. A parameter controller 14a in the 
system controller 14 adjusts the data as described later, 
and inputs the adjusted data of the amplitude ym»= and 
initial phase <Dmn' to the flicker generating block 53 in the 
flicker reducer 25. 

20 The flicker generating block 53 calculates the flicker 

coefficient rn(y), represented by equation (2) of Fig. 17A, 
from the adjusted data of the amplitude ym' and initial 
phase Omn' instead of the amplitude ym and initial phase <t)mn 
obtained from the DFT block 51. In this example, ym and <E>mn 

25 are replaced with ym' and Omn' in equation (2) of Fig. 17A. 
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As shown in Fig. 13, the normalized and integrated 
value calculating block 3 0 in the flicker reducer 25 has the 
same structure as the counterpart in Fig. 4. Alternatively, 
the normalized and integrated value calculating block 3 0 may 
5 have the same structure as the counterpart shown in Fig. 6 
or Fig. 7 . 

<First specific example: Fig. 14> 

Fig. 14 illustrates a first specific example of the 
system controller 14 . 

10 The number of lines for data of the amplitude ym and the 

initial phase Omn from the DFT block 51 are m per field in 
practice, but a single line only is shown. The same is true 
of the output data of the amplitude ymV and the initial 
phase Omn' from the parameter controller 14a. 

15 The data of the amplitude ym and the initial phase Omn 

from the DFT block 51 are fed to digital LPFs (low-pass 
filters) 61 and 62, the output data of the digital LPF 61 is 
fed to a gain adjusting circuit (multiplier circuit) 63, and 
the output data of the gain adjusting circuit 63 is input to 

20 the flicker generating block 53 as adjusted amplitude ym* . 
The output data of the digital LPF 62 is input to the 
flicker generating block 53 as adjusted initial phase Omn' . 

A time constant Ta of the digital LPF 61 and a time 
constant Tp of the digital LPF 62 are set by the time 

25 constant setting block 65. A gain (multiplication 
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coefficient) Ka of the gain adjusting circuit 63 is set by a 
gain setting block 66. 

Preferably, the time constants of the digital LPFs 61 
and 62 are continuously set to any value within a constant 
range. If a desired time constant cannot be set, a time 
constant close to the desired one may be set. If a single 
LPF cannot continuously change the time constant, a 
plurality of LPFs having separate time constants may be 
internally arranged, and by providing time constants Ta and 
Tp as a control code to the LPFs, a single LPF may be 
selected from the plurality of LPFs. 

The initial phase Omn periodically varies during the 
generation of flickering. For example, if the frequency of 
the commercial AC power source is SO Hz and the vertical 
synchronization frequency of the camera is 60 Hz, the 
initial phase <Dmn becomes the same value every three fields, 
and a difference of the initial phase Omn takes place from 
the immediately prior field as represented by .equation (3) 
of Fig. 17A. 

The digital LPF 62 must serve as one LPF for the data 
of the same phase taking into consideration the variation in 
the initial phase Omn. More specifically, if the variation 
period of the initial phase Omn is three fields as described 
above, three LPFs are arranged as the digital LPF 62, and 
the data of the initial phase Omn is distributed among the 
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three LPFs . 

A state detecting block 68 receives the data of the 
amplitude ym and the initial phase Omn, and AE (automatic 
exposure) control information and AWB (automatic white 
5 balance) control information, obtained in the system 
controller 14. More specifically, the AE control 
information is on-screen brightness information, and the AWB 
control information is color temperature information and 
information as to whether the camera is illuminated under a 

10 fluorescent lamp. 

In response to these inputs, the state detecting block 
68 detects the photographing conditions affecting the 
generation of the fluorescent light flicker. More 
specifically, the state detecting block 68 determines 

15 whether a current photographing condition is under the 
illumination of the fluorescent lamp, or a transitional 
state from the illumination of the non- fluorescent lamp to 
the illumination of the fluorescent lamp when the > 
fluorescent lamp is turned on or a transitional state from 

20 the illumination of the fluorescent lamp to the illumination 
of the non- fluorescent lamp when the fluorescent lamp is 
turned off. In response to the determination results, a 
control mode is determined. 

A control mode indicating signal is fed to the time 

25 constant setting block 65 and the gain setting block 66 to 
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indicate the determined control mode thereto. In response, 
the time constant setting block 65 sets the time constants 
Ta and Tp in the digital LPFs 61 and 62, respectively, and 
the gain setting block 66 sets the gain Ka in the gain 
5 adjusting circuit 63. 

Fig. 16 lists determination criteria for the state 
detection of the state detecting block 68. If the 
flickering occurs steadily and regularly under the 
illumination of the fluorescent lamp, the amplitude ym of 

10 the estimated flicker component becomes generally constant, 

and the initial phase Omn takes substantially the same value 
every constant period (every three fields at a commercial 
power source frequency of 50 Hz and a camera vertical 
synchronization frequency of 60 Hz) . 

15 These inputs are sufficient to cause the state 

detecting block 68 to determine that the flicker is 
generated steadily and regularly under the illumination of 
the fluorescent lamp. 

Since the on-screen brightness varies generally 

20 periodically under the illumination of the fluorescent lamp, 
the brightness information for the AE control is sufficient 
to cause the state detecting block 68 to determine that the 
camera is under the illumination of the fluorescent lamp. 
In the AWE control, a light source is estimated from 

25 detected color information, and a determination of whether 
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the light source is a fluorescent lamp is determined based 
on the estimated light source. A determination of whether 
the camera is under the illumination of the fluorescent lamp 
can be performed based on the light source estimating 
5 information for the AWB control. 

The above -referenced information over a plurality of 
past fields is generally checked to enhance detection 
accuracy. 

Upon determining that the flickering takes place 
10 steadily and regularly under the illumination of the 

fluorescent lamp, the state detecting block 68 sets, as the 
control mode, a mode A to be discussed later. 

If no flickering takes place regularly under the 
illumination of the non- fluorescent lamp, the amplitude ym 
15 of the estimated flicker component contains only a noise 

component, and varies randomly in the vicinity of zero, and 
the initial phase <Pmn randomly varies because of the noise. 

Such information is sufficient for the state detecting 
block 68 to determine that no flicker reduction operation is 
20 necessary under the illumination of the non- fluorescent lamp 
From the brightness information for the AE control 
indicating that variations in the screen -brightness have no 
periodicity under the illumination of the non- fluorescent 
lamp, the state detecting block 68 determines that the 
25 camera is under the illumination of the non- fluorescent lamp 
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From the light source estimating information for the AWB 
control, the state detecting block 68 determines that the 
camera is under the illumination of the non- fluorescent lamp. 
In this example, the above-referenced information over 
5 a plurality of past fields is generally checked to enhance 
detection accuracy. 

Upon determining that no flickering takes place 
steadily and regularly under the illumination of the non- 
fluorescent lamp (that no flicker reduction operation is 

10 necessary) , the state detecting block 68 sets, as the 
control mode, a mode B to be discussed later . 

In response to the control mode determined by the state 
detecting block 68, the time constant setting block 65 sets 
the time constants Ta and Tp of the digital LPFs 61 and 62, 

15 and the gain setting block 66 sets the gain Ka of the gain 
adjusting circuit 63. Actual values to be set are 
determined as described below in response to the system 
configuration and requirements to the system. 

The time constant Ta of the digital LPF 61 is discussed 

20 first- As previously discussed, the amplitude ym of the 

estimated flicker component slightly varies in the vicinity 
of zero and is generally constant in both the mode A (in 
which the flickering takes place steadily and regularly 
under the illumination of the fluorescent lamp) .and the mode 

25 B (in which no flickering takes place regularly under the 
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illumination of the non- fluorescent lamp) . If external 
disturbance is applied, the amplitude ym is not constant. 

To make the system robust to the external disturbance, 
the time constant Ta of the digital LPF 61 is preferably set 
5 to be longer. However, if the control mode is transitioned 
from the mode A to the mode B or from the mode B to the mode 
A, the time constant Ta of the digital LPF 61 is preferably 
set to be shorter to achieve a fast tracking feature during 
transitions. 

10 More specifically, the amplitude ym must satisfy two 

mutually contradictory requirements at the same time. The 
method of Fig. 4, Fig. 6, or Fig. 7 provides an algorithm 
originally robust to the external disturbance. 

In practice, a shorter time constant Ta is set with a 

15 view to the tracking feature. Most preferably, dynamic 

control may be introduced to provide a longer time constant 
Ta in either the mode A or the mode B and a shorter time 
constant Ta in a transition from the mode A to the mode B or 
in a transition from the mode B to the mode A. 

20 The time constant Tp of the digital LPF 62 is discussed 

now. During the mode A (when the flickering takes place 
steadily and regularly under the illumination of the 
fluorescent lamp) , the initial phase Omn becomes 
substantially the same value every constant period based on 

25 the principle of the flicker generation as shown in Fig. 16. 
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A sufficiently longer time constant Tp needs to be set to be 
robust to the external disturbance. 

In contrast, the initial phase Omn takes a random 
number during the mode B (when no flickering takes place 
5 regularly under the illumination of the non- fluorescent 
lamp) , and no particular advantage is provided by the 
setting of a longer time constant Tp. More specifically, 
during the mode B, the time constant Tp may be set to any 
value taking into consideration the effect of a gain 
10 adjustment to be discussed later . 

The time constant Ta and the time constant Tp may be 
alternated between the mode A and the mode B. ' 

The gain Ka of the gain adjusting circuit 63 is now 
discussed- During the mode A (when the flickering takes 
15 place steadily and regularly under the illumination of the 
fluorescent lamp) , the amplitude ym is substantially 
constant as shown in Fig. 16, and the gain Ka can be set to 
1- 

The gain Ka determines a correction rate of the 
20 amplitude ym (if a gain Ka=l is set, 100% of the input is 
output with a correction rate at zero) , and the correction 
rate of the amplitude ym is directly controlled by varying 
the gain Ka. 

Under actual photographic environments, the amplitude 
25 may be intentionally enlarged or intentionally decreased. 
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For this reason, the gain Ka is not limited to 1, and the 
system may be designed to set the gain Ka to be larger than 
1 or smaller than 1 . 

During the mode B (in the state that no flicker is 
,5 generated steadily under the illumination of the non- 
fluorescent lamp) , the amplitude ym takes a random value in 
the vicinity of zero in response to noise. Since no flicker 
reduction operation is required by nature during the mode B, 
the gain Ka is set to zero to disable the unnecessary 
10 process . 

The mode A and the .mode B as the steady state (the 
state that the flicker is steadily generated or the state 
that no flicker is steadily generated) have been discussed. 
If the mode detected by the state detecting block 68 is 

15 transitioned from the mode A to the mode B, it is likely 

that the photographic environment has changed from under the 
illumination of the fluorescent lamp to under the 
illumination of the non- fluorescent lamp. If the mode 
detected by the state detecting block 68 is transitioned 

20 from the mode B to the mode A, it is likely that the 
photographic environment has changed from under the 
illumination of the non- fluorescent lamp to under the 
illumination of the fluorescent lamp. 

As previously discussed with reference to Fig. 4, Fig. 

25 6, or Fig. 1, in accordance with the basic method of the 
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present invention, the flicker component is extracted 
through the averaging operation or the difference operation 
throughout a plurality of fields. During the transition, 
one portion of a signal string for use in the averaging 
5 operation or the difference operation contains a flicker 
component and another portion of the signal string has no 
flicker component . 

As a result, an error takes place in the flicker 
component obtained through the averaging operation or the 

10 difference operation, and an error also takes place in the 

detected amplitude ym and initial phase Omn. If the flicker 
coefficient Tn (y) is calculated from the error-affected 
amplitude ym and initial phase Omn, the output image signal 
is also adversely affected. 

15 To alleviate this problem, the gain setting block 66 

detects a transitional state and controls the gain Ka in 
response to the transitional state. 

More specifically, if the control mode is shifted from 
the mode A to the mode B, the reliability = of each of the 

20 amplitude ym and the initial phase Omn has dropped at the 
start of the transition. Immediately subsequent to the 
transition, the gain Ka is switched from 1 to zero, and the 
flicker reduction operation of the flicker generating block 
53 and the calculating block 40 is stopped. Alternatively, 

25 the gain Ka. is gradually lowered, and the flicker reduction 
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operation of the flicker generating block 53 and the 
calculating block 40 is smoothly stopped. 

Conversely, If the control mode is transitioned from 
the mode B to the mode A, the reliability of each of the 
5 amplitude ym and the initial phase Omn is still low at the • 
start of the transition. After time elapse until the 
reliability of each of the amplitude ym and the initial 
phase Omn rises to a sufficiently high level, the flicker 
generating block 53 and the calculating block 4 0 performs 

10 the flicker reduction operation. Alternatively, the gain Ka 
is gradually raised so that the flicker generating block 53 
and the calculating block 40 smoothly perform the flicker 
reduction operation. 

<Second specific example: Fig. 15> 

15 Fig. 15 illustrates a second specific example of the 

system controller 14 . 

In this example, memories 71-74, switches ' 75-78 , and a 
state detecting block 69 are added to the example of Fig. 14. 
The memory 71 stores the data of the amplitude ym, the 

20 memory 72 stores the data of the initial phase Omn, the 
memory 73 stores the output data of the gain adjusting 
circuit 63, and the memory 74 stores the output data of the 
digital LPF 62. The switches 75-78 select the input data 
and the output data of the memories 71-74, respectively, in 

25 response to the detection results of the state detecting 
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block 69. The output data of the switch 75 is fed to the 
digital LPF 61, the output data of the switch 76 is fed to 
the digital LPF 62, the output data of the switch 77 is fed 
to the flicker generating block 53 as the data of the 
5 amplitude ym' , and the output data of the switch 7 8 is fed 
to the flicker generating block 53 as the data of the 
initial phase Omn' . 

The state detecting block 69 receives zooming 
information and hand- shake information. In response to the 

10 zooming information, the state detecting block 69 determines 
whether a large variation in the angle of view takes place 
in response to a zooming action, and in response to the 
hand-shake information, the state detecting block 69 
determines whether a large variation in the angle of view 

15 takes place in response to panning and tilting actions and a 
large -amplitude hand shake. 

If it is determined that no large variations take place 
in the angle of view, the state detecting block 69 sets the 
switches 75-78 to the sides thereof opposite from the sides 

20 of the memories 71-74. Normally, the flicker reduction 
operation is performed as in the example of Fig. 14. 

If it is determined that a large variation takes place 
in the angle of view, the state detecting block 69 sets the 
switches 75-78 to the sides thereof to the memories 71-74. 

25 Since the reliability of each of the amplitude ym and 
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the initial phase Omn drops when a large variation occurs in 
the angle of view, amplitude data and initial phase data 
acquired in the past and respectively stored in the memory 
73 and the memory 74 are input to the flicker generating 
5 block 53 as the amplitude ym* and the initial phase Omn* . 

Since the amplitude ym and the initial phase Omn are 
steady during the mode A (in the state that the flickering 
is generated steadily and regularly under the illumination 
of the fluorescent lamp) as shown in Fig. 16, the use of the 

10 past data presents no problem at all. On the contrary, the • 
past data should be positively used. 

However, if low-reliability data is successively input 
to the digital LPFs 61 and 62 in the middle of variation in 
the angle of view, the amplitude ym' and the initial phase 

15 Omn' obtained immediately subsequent to the setting of the 
switches 77 and 78 to the sides thereof opposite from the 
memories 73 and 74 suffer from error. 

To eliminate .such a problem, during the occurrence of a. 
large variation in the angle of view, the state detecting 

20 block 69 not only sets the switches 77 and 78 to the sides 
thereof to the memiories 73 and 74, respectively, but also 
sets the switches 75 and 76 to the sides thereof to the 
memories 71 and 72, respectively. With this arrangement, 
the low-reliability data are not input to the digital LPFs 

25 61 and 62, and high-reliability data obtained prior to the 
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occurrence of the large variation in the angle of view and 
stored in the memories 71 and 72 are input to the digital 
LPFs 61 and 62. 

Regardless of the zooming information and the hand- 
5 shake information, the reliabilities of the amplitude ym and 
the initial phase Omn are separately determined. 
Reliability level information as a result of determination 
is input to the state detecting block 69. If the 
reliability level information indicates that 'the reliability 
10 of the amplitude ym and the initial phase 'Omn is low, the 

switches 75-78 are set to the sides thereof to the memories 
71-74. In this way, past data having a high reliability is 
used. 

<Advantages> 

15 In accordance with the above -referenced embodiments, 

the flicker reduction operation is less sensitive to the 
.external disturbance under the illumination of the 
fluorescent lamp and under the illumination of the non- 
fluorescent lamp, and still provides fast response and 

20 tracking features during the transitional state . The image 
pickup device performs the proper flicker reduction 
operation smoothly without discordance at the moment of the 
state transition, at the occurrence of the variation in the 
angle of view, or when the flicker detection parameter is 

25 low in reliability. 
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[Alternate embodiments] 
(Integration) 

In each of the above -referenced examples, the input 
image signal In' (x,y) is integrated throughout one line. . 
5 Since the input image signal In*{x,y) is integrated to 
obtain a sample value of the flicker component with the 
effect of a pattern of the image reduced, the integration 
may be performed throughout a plurality of lines rather than 
only one line. One period of the fluorescent light flicker 

10 (on-screen flicker) appearing as a striped pattern on the 
screen corresponds to L (=M*60/100) lines as previously 
described. If at least two sample values are obtained in 
one period, i.e., L lines, the flicker component is detected 
based on the sampling theorem. 

15 In practice, a plurality of sample values, for example, 

at least 10 sample values are preferably obtained from one 
period, i.e., L lines of the on-screen flicker.. Even in 
this case, the horizontal period of the input image signal 
In' (x,y) is integrated over a duration of time equal to or 

20 more than several times the horizontal periods, more 

particularly, ten times longer than the horizontal period. 
The integration time is not limited to an integer multiple 
of horizontal periods, and for example, 2.5 .times the 
horizontal period is perfectly acceptable. 

25 If the integration time is prolonged and a sample count 
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per unit time is reduced, workload involved in the DFT 
operation by the DFT block 51 is lightened. When a subject 
moves in a vertical direction of the screen, the effect of 
the motion is reduced. 
5 (Other alternate embodiments) 

In the primary color system of Fig, 2, the flicker . 
components in the RGB primary color signals is detected and 
reduced by the flicker reducers 25R, 25G, and 25B on a per 
color signal basis. Alternatively, the previously described 

10 flicker reducer 25 is arranged on the output side of the 
luminance signal Y of the synthesis matrix circuit 29 in 
order to detect and reduce the flicker component in the 
luminance signal Y. 

In accordance with the above -referenced embodiments, 

15 the digital signal processor 20 including the flicker 

reducer 25 is implemented by hardware. Part or whole of the 
flicker reducer 25 or the digital signal processor 20 may be 
implemented by software . 

In accordance with the above -referenced embodiments, 

20 the vertical synchronization frequency is 6 0 Hz (with one 
field period equal to 1/60 second) . The present invention 
is applicable to a progressive type camera having a vertical 
synchronization frequency of 3 0 Hz (with one frame period 
being 1/3 0 second) . In this case, three frame period (l/lO 

25 second) is an integer multiple (the stripe pattern of the 
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flicker is 10 wavelengths over three frames) of the emission 
period of (1/100 second) of a fluorescent lamp, and the 
field of the above -referenced embodiment may be substituted 
for the frame . 

The present invention is applicable to an XY addressing 
type image pickup element such as an image pickup element 
other than a CMOS image pickup element . 

Industrial Applicability 

In accordance with the present invention, the flicker 
unique to the fluorescent lamp appearing in the XY 
addressing type image pickup element such as a CMOS image 
pickup element is detected and reduced reliably and 
sufficiently through the simple signal processing without 
using a -photosensitive element regardless of the subject, 
the video signal level, and the type of the fluorescent lamp. 

In particular, when the flicker reduction operation of 
the present invention is used for the RGB primary color 
signals, not only the light and dark flicker but also the 
color flicker is accurately detected, and reliably and 
sufficiently reduced. 

If the level of the signal falls within the saturation 
region, the flicker reduction is disabled. In this way, the 
signal component is prevented from being affected by the 
flicker reduction operation. 
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The flicker reduction operation is disabled under the 
illumination of the non- fluorescent lamp. In this way, the 
effect of the flicker reduction operation on the image 
quality is controlled. 
5 If the subject changes greatly in a short period of 

time in response to the operation or the action of the 
photographer, the immediately precedingly estimated flicker 
component or the flicker component estimated from the 
immediately preceding signal is calculated. In this 

10 arrangement, flicker detection accuracy is free from 

degradation due a large change in the subject happening in a 
short period of time. 

The flicker reduction operation is disabled in the 
photographing condition that does not need the flicker 

15 reduction operation, for example, in the still image taking. 
In this way, the effect of the flicker reduction operation 
on the image quality is controlled. 

The estimated flicker component is adjusted, and the 
adjusted flicker component and the input image signal are 

20 calculated. With this arrangement, the signal component is 
robust to the effect of external disturbance under the 
illumination of the fluorescent lamp and under the 
illumination of the non- fluorescent lamp. The image pickup 
device achieves excellent response and tracking features at 

25 the transition. Furthermore, the image pickup device 
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performs the processes smoothly without discordance at the 
state transitions, at the variation in the angle of view, or 
when the reliability of the flicker detection parameters .is 
low. 



